Method of processing image of electronic device and electronic device thereof

ABSTRACT

According to various embodiments of the present disclosure, an electronic device includes: a camera module that obtains an image; and a processor which implements the method, including setting a quadrangular area in the obtained image including a reference pixel and a corresponding pixel disposed respectively at corners of the quadrangular area, calculating an accumulated-pixel value for each pixel of the obtained image corresponding to the quadrangular area, such that a particular pixel value for a particular pixel is a sum of the pixel values beginning from the reference pixel, continuing though an arrangement of pixels in the quadrangular area and terminating at the particular pixel, and generating an image quality processing-dedicated frame of accumulated-pixel values based on calculated accumulated-pixel values of each pixel of the frame.

CLAIM OF PRIORITY

This application claims the priority under 35 U.S.C. §119(a) to KoreanApplication Serial No. 10-2015-0140305, which was filed in the KoreanIntellectual Property Office on Oct. 6, 2015, the entire content ofwhich is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to an image processing method and anelectronic device thereof.

BACKGROUND

Generally, a digital signal processor (DSP) is a hardware device thatperforms image processing. For example, the DSP may execute “3A”processing (e.g., auto exposure or “AE”, auto white balance or “AWB”,and auto focus or “AF”), to improve the quality of images capturedthrough a digital camera. To do so, the DSP may set a predetermined areawithin an image, sometimes referred to as a ‘block’ (or a ‘region ofinterest’ or “ROI”). For example, a single block may be set for AFprocessing in an image. Similarly plurality of blocks may be set for AWBprocessing or AE processing in an image. Subsequently, the DSP mayexecute image processing on the set block to generate image qualityprocessing-dedicated data on image data in the block.

SUMMARY

Generally, an image obtained through a camera may be distinguished basedon a vertical drive (VD) signal generated from an image sensor. FIG. 15Amay show a single block for AF processing, whereas FIG. 15B may show aplurality of blocks for AWB or AE processing. Also, image processingexecuted on the captured image may be performed in the order ofoperations in which the DSP sets a block, the image sensor receiveslight, and the DSP performs image processing on the image correspondingto the part where the block is set. For example, as illustrated in FIG.16, image quality processing-dedicated data is obtained based on thesummation of pixel values of a previously set block part in an imagethat is obtained by an image sensor and is data-processed.

When the amount of time expended for performing the operations (e.g.,light exposure, data processing, and image quality processing-dedicateddata obtaining) to obtain the image quality processing-dedicated data is1VD, at least 2VD time is generally expended to perform operations fromsetting a block to obtaining, from an image obtained through a camera,image quality processing-dedicated data of a part in which the block isset, according to FIG. 17.

A recent technological trend in enhancing camera image quality includespromptly predicting the movement of a subject, and generating an imagebased on a corresponding algorithm. However, setting a block, receivinglight through an image sensor, and performing data processing on animage where the block is set requires at least 2VD time, and thus, alarge amount of time is spent performing image processing, which is adisadvantage.

Also, when a fixed block is set in an image, image data may be extractedfrom the fixed area, and thus, it is difficult to embody a flexiblealgorithm associated with an image quality. Referring to FIG. 18, in oneexample, a single image processing operation for extracting imagequality processing-dedicated data may extract image qualityprocessing-dedicated data associated with a block 181 that is set inadvance, but may not extract image quality processing-dedicated dataassociated with areas 183, 185, and 187 in addition to the block 181that is set in advance, unlike the example of the bottom figure of FIG.18. For example, when the reliability of image qualityprocessing-dedicated data extracted from a predetermined block is low,or when image quality processing-dedicated data extracted through analgorithm is undesirable image quality processing-dedicated data, imagequality processing on an image may be processed through the data of anadjacent area of the block. However, a system that sets a fixed blockmay not be capable of performing the above processing.

Also, in one example, 3A data obtained according to 3A processing of theDSP is a value corresponding to a block part set in advance. To obtainimage quality processing-dedicated data of another block part, the otherblock part needs to be set and at least 2VD time is utilized to obtainimage quality processing-dedicated data, which is a drawback from theperspective of time. Therefore, the DSP does not allow the frequentchanging of a block that is set, and thus, the algorithm for 3Aprocessing is designed to be inflexible, which is a drawback.

Accordingly, various embodiments of the present disclosure provide animage processing method and an electronic device thereof, which iscapable of obtaining 3A data without setting a block of an imageobtained through a camera before extracting image qualityprocessing-dedicated data of the image, so as to reduce the amount oftime expended for image processing.

Also, various embodiments of the present disclosure provide a flexiblealgorithm for enhancing the image quality of an image obtained through acamera, so as to apply various image quality algorithms, and thus,increase the image quality of the image obtained through the camera.

According to embodiments of the present disclosure, an electronic deviceis provided including a camera module for obtaining an image and aprocessor to perform: setting a quadrangular area in the obtained imageincluding a reference pixel and a corresponding pixel disposedrespectively at corners of the quadrangular area, calculating anaccumulated-pixel value for each pixel of the obtained imagecorresponding to the quadrangular area, such that a particular pixelvalue for a particular pixel is a sum of the pixel values beginning fromthe reference pixel, continuing though an arrangement of pixels in thequadrangular area and terminating at the particular pixel, andgenerating an image quality processing-dedicated frame ofaccumulated-pixel values based on calculated accumulated-pixel values ofeach pixel of the image.

According to embodiments of the present disclosure, an image processingmethod includes obtaining an image through a camera module of theelectronic device, setting a quadrangular area in the obtained imageincluding a reference pixel and a corresponding pixel disposedrespectively at corners of the quadrangular area, calculating anaccumulated-pixel value for each pixel of the obtained imagecorresponding to the quadrangular area, such that a particular pixelvalue for a particular pixel is a sum of the pixel values beginning fromthe reference pixel, continuing through an arrangement of pixels in thequadrangular area and terminating at the particular pixel, andgenerating an image quality processing-dedicated of accumulated-pixelvalues based on calculated accumulated-pixel values of each pixel of theimage.

According to embodiments of the present disclosure, an electronic deviceis disclosed including a camera module that obtains an image, and aprocessor, configured to perform: generating an image qualityprocessing-dedicated frame with respect to the obtained image based onthe obtained image, and setting at least one of: a first region ofinterest disposed within the image using the generated image qualityprocessing-dedicated image, and a second region of interest having asize or disposed at a location differing from at least one of a size andlocation of the first region of interest, respectively.

According to embodiments of the present disclosure, an image processingmethod is disclosed, including obtaining an image through a cameramodule of the electronic device, generating by a processor an imagequality processing-dedicated image with respect to the obtained imagebased on the obtained image, and setting at least one of: a first regionof interest disposed within the image using the generated image qualityprocessing-dedicated image, and a second region of interest having asize or disposed at a location differing from at least one of a size andlocation of the first region of interest, respectively.

An image processing method and an electronic device thereof, accordingto various embodiments of the present disclosure, may obtain 3A datawithout setting a block with respect to an image obtained through acamera, and thus, may overcome a problem in which at least 2VD time isutilized to perform image processing. For example, an image qualityprocessing-dedicated frame is generated for each frame of an imageobtained through a camera, and thus, a series of operations forobtaining image quality processing-dedicated data of a shot image withrespect to a block to be generated do not need to be performed everytime that a block is newly generated (or every time that at least one ofthe location or the size of a block is changed). According to variousembodiments of the present disclosure, image qualityprocessing-dedicated data with respect to a block to be generated inassociated with an image may be readily obtained by using an imagequality processing-dedicated frame that is generated in advance. Throughthe advantage from the perspective of time, when an operation ofenhancing the image quality of an image obtained through a camera isexecuted by tracing a moving subject, a block may be promptly predictedand the accuracy of image quality processing on an image may beimproved.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects and features of the present disclosure willbe more apparent from the following detailed description taken inconjunction with the accompanying drawings, in which:

FIG. 1 illustrates a network environment including an electronic deviceaccording to various embodiments of the present disclosure;

FIG. 2 is a block diagram of an electronic device according to variousembodiments of the present disclosure;

FIG. 3 is a block diagram of a programming module according to variousembodiments of the present disclosure;

FIG. 4 is a flowchart illustrating an image processing operation of anelectronic device according to various embodiments of the presentdisclosure;

FIG. 5A and FIG. 5B are diagrams illustrating an image processingoperation of an electronic device according to various embodiments ofthe present disclosure;

FIG. 6 is a diagram illustrating a method of obtaining data of apredetermined area from an image quality processing-dedicated frameaccording to various embodiments of the present disclosure;

FIG. 7 is a flowchart illustrating an image processing operation of anelectronic device according to various embodiments of the presentdisclosure;

FIG. 8 is a flowchart illustrating an image processing operation of anelectronic device according to various embodiments of the presentdisclosure;

FIG. 9 is a diagram illustrating a basic image qualityprocessing-dedicated frame generated by an image processing operation ofan electronic device according to various embodiments of the presentdisclosure;

FIG. 10 is a diagram illustrating a basic image qualityprocessing-dedicated frame of which the size is reduced by an imageprocessing operation of an electronic device according to variousembodiments of the present disclosure;

FIG. 11, FIG. 12, and FIG. 13 are diagrams illustrating an example ofusing a basic image quality processing-dedicated frame, which isgenerated by an image processing operation of an electronic deviceaccording to various embodiments of the present disclosure;

FIG. 14 is a diagram illustrating an operating time of an imageprocessing operation of an electronic device according to variousembodiments of the present disclosure;

FIG. 15A and FIG. 15B are diagrams illustrating a block that is set inan image;

FIG. 16 is a flowchart illustrating an image processing operation of anelectronic device;

FIG. 17 is a diagram illustrating an operating time of an imageprocessing operation of an electronic device; and

FIG. 18 is a diagram illustrating a block that is capable of being setin an image.

DETAILED DESCRIPTION

Hereinafter, various embodiments of the present disclosure will bedescribed with reference to the accompanying drawings. However, itshould be understood that there is no intent to limit the presentdisclosure to the particular forms disclosed herein; rather, the presentdisclosure should be construed to cover various modifications,equivalents, and/or alternatives of embodiments of the presentdisclosure. In describing the drawings, similar reference numerals maybe used to designate similar constituent elements.

As used herein, the expression “have”, “may have”, “include”, or “mayinclude” refers to the existence of a corresponding feature (e.g.,numeral, function, operation, or constituent element such as component),and does not exclude one or more additional features.

In the present disclosure, the expression “A or B”, “at least one of Aor/and B”, or “one or more of A or/and B” may include all possiblecombinations of the items listed. For example, the expression “A or B”,“at least one of A and B”, or “at least one of A or B” refers to all of(1) including at least one A, (2) including at least one B, or (3)including all of at least one A and at least one B.

The expression “a first”, “a second”, “the first”, or “the second” usedin various embodiments of the present disclosure may modify variouscomponents regardless of the order and/or the importance but does notlimit the corresponding components. For example, a first user device anda second user device indicate different user devices although both ofthem are user devices. For example, a first element may be termed asecond element, and similarly, a second element may be termed a firstelement without departing from the present disclosure.

It should be understood that when an element (e.g., first element) isreferred to as being (operatively or communicatively) “connected,” or“coupled,” to another element (e.g., second element), it may be directlyconnected or coupled directly to the other element or any other element(e.g., third element) may be interposer between them. In contrast, itmay be understood that when an element (e.g., first element) is referredto as being “directly connected,” or “directly coupled” to anotherelement (second element), there are no element (e.g., third element)interposed between them.

The expression “configured to” used in the present disclosure may beexchanged with, for example, “suitable for”, “having the capacity to”,“designed to”, “adapted to”, “made to”, or “capable of” according to thesituation. The term “configured to” may not necessarily imply“specifically designed to” in hardware. Alternatively, in somesituations, the expression “device configured to” may mean that thedevice, together with other devices or components, “is able to”. Forexample, the phrase “processor adapted (or configured) to perform A, B,and C” may mean a dedicated processor (e.g., embedded processor) forperforming the corresponding operations or a generic-purpose processor(e.g., central processing unit (CPU) or application processor (AP)) thatcan perform the corresponding operations by executing one or moresoftware programs stored in a memory device.

The terms used in the present disclosure are used to describe specificembodiments, and are not intended to limit the present disclosure. Asingular expression may include a plural expression unless they aredefinitely different in a context. Unless defined otherwise, all termsused herein, including technical and scientific terms, have the samemeaning as those commonly understood by a person skilled in the art towhich the present disclosure pertains. Such terms as those defined in agenerally used dictionary may be interpreted to have the meanings equalto the contextual meanings in the relevant field of art, and are not tobe interpreted to have ideal or excessively formal meanings unlessclearly defined in the present disclosure. In some cases, even the termdefined in the present disclosure should not be interpreted to excludeembodiments of the present disclosure.

An electronic device according to various embodiments of the presentdisclosure may include at least one of, for example, a smart phone, atablet Personal Computer (PC), a mobile phone, a video phone, anelectronic book reader (e-book reader), a desktop PC, a laptop PC, anetbook computer, a workstation, a server, a Personal Digital Assistant(PDA), a Portable Multimedia Player (PMP), a MPEG-1 audio layer-3 (MP3)player, a mobile medical device, a camera, and a wearable device.According to various embodiments, the wearable device may include atleast one of an accessory type (e.g., a watch, a ring, a bracelet, ananklet, a necklace, a glasses, a contact lens, or a Head-Mounted Device(HMD)), a fabric or clothing integrated type (e.g., an electronicclothing), a body-mounted type (e.g., a skin pad, or tattoo), and abio-implantable type (e.g., an implantable circuit).

According to some embodiments, the electronic device may be a homeappliance. The home appliance may include at least one of, for example,a television, a Digital Video Disk (DVD) player, an audio, arefrigerator, an air conditioner, a vacuum cleaner, an oven, a microwaveoven, a washing machine, an air cleaner, a set-top box, a homeautomation control panel, a security control panel, a TV box (e.g.,Samsung HomeSync™, Apple TV™, or Google TV™, a game console (e.g., Xbox™and PlayStation™), an electronic dictionary, an electronic key, acamcorder, and an electronic photo frame.

According to another embodiment, the electronic device may include atleast one of various medical devices (e.g., various portable medicalmeasuring devices (a blood glucose monitoring device, a heart ratemonitoring device, a blood pressure measuring device, a body temperaturemeasuring device, etc.), a Magnetic Resonance Angiography (MRA), aMagnetic Resonance Imaging (MRI), a Computed Tomography (CT) machine,and an ultrasonic machine), a navigation device, a Global PositioningSystem (GPS) receiver, an Event Data Recorder (EDR), a Flight DataRecorder (FDR), a Vehicle Infotainment Devices, an electronic devicesfor a ship (e.g., a navigation device for a ship, and a gyro-compass),avionics, security devices, an automotive head unit, a robot for home orindustry, an automatic teller's machine (ATM) in banks, point of sales(POS) in a shop, or internet device of things (e.g., a light bulb,various sensors, electric or gas meter, a sprinkler device, a firealarm, a thermostat, a streetlamp, a toaster, a sporting goods, a hotwater tank, a heater, a boiler, etc.).

According to some embodiments, the electronic device may include atleast one of a part of furniture or a building/structure, an electronicboard, an electronic signature receiving device, a projector, andvarious kinds of measuring instruments (e.g., a water meter, an electricmeter, a gas meter, and a radio wave meter). The electronic deviceaccording to various embodiments of the present disclosure may be acombination of one or more of the aforementioned various devices. Theelectronic device according to some embodiments of the presentdisclosure may be a flexible device. Further, the electronic deviceaccording to an embodiment of the present disclosure is not limited tothe aforementioned devices, and may include a new electronic deviceaccording to the development of technology.

Hereinafter, an electronic device according to various embodiments willbe described with reference to the accompanying drawings. In the presentdisclosure, the term “user” may indicate a person using an electronicdevice or a device (e.g., an artificial intelligence electronic device)using an electronic device.

An electronic device 101 in a network environment 100 according tovarious embodiments will be described with reference to FIG. 1. Theelectronic device 101 may include a bus 110, a processor 120, a memory130, an input/output interface 150, a display 160, a communicationinterface 170, a camera module 180, and a digital signal processor (DSP)190. In some embodiments of the present disclosure, the electronicdevice 101 may omit at least one of the components, or may furtherinclude other components.

The bus 110 may include, for example, a circuit that interconnects thecomponents 110 to 170 and delivers communication (e.g., a controlmessage and/or data) between the components 110 to 170.

The processor 120 may include one or more of a central processing unit(CPU), an application processor (AP), and a communication processor(CP). For example, the processor 120 may control at least one othercomponent of the electronic device 101 and/or may carry out operationsor data processing related to communication.

The memory 130 may include a volatile memory and/or a non-volatilememory. The memory 130 may store, for example, commands or data relatedto one or more other components of the electronic device 101. Accordingto an embodiment of the present disclosure, the memory 130 may storesoftware and/or a program 140. The program 140 may include, for example,a kernel 141, middleware 143, an application programming interface (API)145, and/or application programs (or “applications”) 147. At least someof the kernel 141, the middleware 143, and the API 145 may be referredto as an Operating System (OS).

The kernel 141 may control or manage system resources (e.g., the bus110, the processor 120, the memory 130, or the like) used for executingan operation or function implemented by other programs (e.g., themiddleware 143, the API 145, or the application(s) 147). Furthermore,the kernel 141 may provide an interface through which the middleware143, the API 145, or the application programs 147 may access individualcomponents of the electronic device 101 to control or manage systemresources.

The middleware 143 may function as, for example, an intermediary forallowing the API 145 or the application programs 147 to communicate withthe kernel 141 to exchange data.

Further, the middleware 143 may process one or more task requestsreceived from the application programs 147 according to prioritiesthereof. For example, the middleware 143 may assign priorities to atleast one of the application programs 147 in association with usingsystem resources (e.g., the bus 110, the processor 120, the memory 130or the like) of the electronic device 101. For example, the middleware143 may perform scheduling, load balancing, or the like on the one ormore task requests by processing the one or more task requests accordingto the priorities assigned thereto.

The API 145 is an interface through which the applications 147 controlfunctions provided from the kernel 141 or the middleware 143, and mayinclude, for example, at least one interface or function (e.g.,instruction) for file control, block control, image processing, textcontrol, or the like.

For example, the input/output interface 150 may serve as an interfacethat may transfer commands or data input from a user or another externaldevice to other component (s) of the electronic device 101. Furthermore,the input/output interface 150 may output the commands or data receivedfrom the other component(s) of the electronic device 101 to a user oranother external device.

The display 160 may include, for example, a liquid crystal display(LCD), a light emitting diode (LED) display, an organic light emittingdiode (OLED) display, a micro electro mechanical system (MEMS) display,or an electronic paper display. The display 160 may display, forexample, various types of contents (e.g., text, images, videos, icons,or symbols) to a user. The display 160 may include a touch screen andreceive, for example, a touch, gesture, proximity, or hovering inputusing an electronic pen or a user's body part.

The communication interface 170, for example, may set communicationbetween the electronic device 101 and an external device (e.g., aexternal electronic device 102, an external electronic device 104, or aserver 106). For example, the communication interface 170 may beconnected to a network 162 through wireless or wired communication tocommunicate with an external device (e.g., the second externalelectronic device 104 or the server 106).

The wireless communication may use at least one of, for example, longterm evolution (LTE), LTE-Advance (LTE-A), code division multiple access(CDMA), wideband CDMA (WCDMA), universal mobile telecommunicationssystem (UMTS), WiBro (Wireless Broadband), and global system for mobilecommunications (GSM), as a cellular communication protocol. Further, thewireless communication may include, for example, short-rangecommunication 164. The short-range communication 164 may include, forexample, at least one of Wi-Fi, Bluetooth, near field communication(NFC), global navigation satellite system (GNSS), and the like. The GNSSmay include at least one of, for example, a global positioning system(GPS), a global navigation satellite system (Glonass), a Beidounavigation satellite system (hereinafter, referred to as “Beidou”), andGalileo (European global satellite-based navigation system).Hereinafter, in the present disclosure, the “GPS” may be interchangeablyused with the “GNSS”. The wired communication may include at least oneof, for example, a universal serial bus (USB), a high definitionmultimedia interface (HDMI), Recommended Standard 232 (RS-232), and aplain old telephone service (POTS). The network 162 may include at leastone of a telecommunication network, such as a computer network (e.g., aLAN or a WAN), the Internet, and a telephone network.

Each of the first and second external electronic devices 102 and 104 maybe of a type that is identical to, or different from, that of theelectronic device 101. According to an embodiment of the presentdisclosure, the server 106 may include a group of one or more servers.According to various embodiments of the present disclosure, all or someof the operations performed in the electronic device 101 may beperformed in another electronic device or a plurality of electronicdevices (e.g., the electronic devices 102 and 104 or the server 106).According to an embodiment of the present disclosure, when theelectronic device 101 should perform some functions or servicesautomatically or by request, the electronic device 101 may make arequest for performing at least some of the functions related to thefunctions or services to another device (e.g., the electronic device 102or 104 or the server 106) instead of, or in addition to, performing thefunctions or services by itself The other electronic device (e.g., theelectronic device 102 or 104, or the server 106) may carry out therequested functions or additional functions and transfer the resultthereof to the electronic device 101. The electronic device 101 mayprovide the received result as it is or additionally process thereceived result and provide the requested functions or services. To thisend, for example, cloud computing, distributed computing, orclient-server computing technology may be used.

The camera module 180 may obtain an image. For example, the cameramodule 180 may shoot a still image and a video. For example, the cameramodule 180 may include an optical unit (not illustrated), a lens drivingunit (not illustrated), an image sensor (not illustrated), andCDS/AGC/ADC (not illustrated).

The optical unit may include a lens (not illustrated) and a shutter (notillustrated). The optical unit may be driven by the lens driving unitand may shoot an image. Also, the lens of the optical unit is driven bythe lens driving unit, and may perform zooming-in, focusing, and thelike.

The image sensor may sense an image shot by the optical unit, and mayconvert the same into an electrical signal. The image sensor may be acomplementary metal-oxide-semiconductor (CMOS) or a charge coupleddevice (CCD) sensor, and may be a sensor that is capable of sensing animage that is higher than or equal to the ultra high definition (UHD)level.

The CDS/AGC/ADC removes noise from an output signal of the image sensorusing a correlated double sampling circuit (CDS), adjusts a gain tomaintain the level of a signal to be constant using an auto gaincontrolling circuit (AGC), and converts an analog signal into a digitalsignal using an analog to digital converter (ADC).

The DSP 190 may reduce the size of an image obtained through the cameramodule 180. For example, through techniques, such as sub-sampling,binning, or the like, the size of the image obtained through the cameramodule 180 may be reduced.

The sub-sampling is an image processing technique that converts an imageformed of pixels, provided in a predetermined size, into a smaller size.Also, binning is an image processing technique that obtains a highquality image through the summation of the pixel values of adjacentpixels in the image.

The DSP 190 may perform data processing on the obtained image, so as togenerate image quality data with respect to the image (or the image ofwhich the size is reduced) of the obtained image.

The image quality data may include, for example, AF data for auto focus(AF), AWB data for auto white balance (AWB), or AE data for autoexposure (AE).

For example, the data processing may filter the obtained image, mayextract the edge part of a subject, and may generate an image, so as togenerate image quality data associated with AF data. Also, for example,the data processing may perform pre-processing, such as dividing theobtained image into a designated number of parts, or the like, so as togenerate image quality data associated with AWB data or AE data.

The AF data may be, for example, a value that may be obtained byfiltering the obtained image, extracting an edge part, and adding up thevalues of the extracted edge part. Also, the AWB data may be, forexample, an average value of each of R, G, and B, which is calculated byadding up the pixels of the obtained image. Also, the AE data may be,for example, an average value of each of R, G, and B, which iscalculated by adding up the pixels of the obtained image.

When the AWB data or the AE data is generated, data processing on theobtained image may be omitted based on the performance, the capacitance,and the like of the DSP 190.

The DSP 190 may generate an image quality processing-dedicated frame ofthe frame (or a frame of which the size is reduced or a data-processedframe) of the obtained image. For example, the DSP 190 may calculate,for each pixel of the obtained image, an accumulated pixel value byadding up the pixel values of pixels in a quadrangular area (alsoreferred to as a block or a region of interest) that takes acorresponding pixel and a reference pixel of the obtained image ascorners. Also, the DSP 190 may generate an image qualityprocessing-dedicated frame including the calculated accumulated-pixelvalue corresponding to each pixel of the obtained image. All of theaccumulated pixel values included in the image qualityprocessing-dedicated frame may be regarded as image qualityprocessing-dedicated data. Accordingly, it is construed that the DSP 190generates image quality processing-dedicated data of the obtained image,so as to generate the image quality data.

When the size of the generated image quality processing-dedicated frameis large, the size of a memory inside the electronic device increases,which is a drawback. To solve the problem, the DSP 190 may reduce thesize of the generated image quality processing-dedicated frame. Forexample, the size of the generated image quality processing-dedicatedframe may be reduced using a technique, such as sub-sampling, binning,or the like.

According to an embodiment of the present disclosure, the DSP 190 mayobtain the data of a first quadrangular area (also referred to as aregion of interest or a block) including at least one pixel, from thegenerated image quality processing-dedicated frame (or image qualityprocessing-dedicated frame of which the size is reduced).

The operation of obtaining the data of the first quadrangular area maybe performed through, for example, adding a second pixel that is incontact in the diagonal direction with a pixel that utilizes, as aboundary, a second corner disposed in the diagonal direction to a firstcorner in the first quadrangular area, from among the pixels external tothe first quadrangular area, to a pixel value of a first pixel in thefirst quadrangular area that utilizes the first corner, which isfarthest from the reference pixel in the first quadrangular area, as aboundary; subtracting, from the sum, a third pixel that is in contact inthe diagonal direction with a pixel that utilizes, as a boundary, athird corner in the first quadrangular area, from among the pixelsoutside the first quadrangular area; and subtracting, from the sum, afourth pixel that is in contact in the diagonal direction with a pixelthat utilizes, as a boundary, a fourth corner in the first quadrangulararea, from among the pixels outside the first quadrangular area.

By using the method of obtaining the data of the first quadrangulararea, image processing for enhancing the image quality of apredetermined area of the obtained image may be readily performed. Forexample, the obtained data of the first quadrangular area may have avalue that is identical to a value obtained by adding up all of thepixel values in a quadrangular area of the obtained image correspondingto the first quadrangular area. Accordingly, the value obtained byadding up the pixel values of a predetermined area of the obtained imagemay be readily obtained. Also, an average value of the pixel values ofthe obtained image, which corresponds to the first quadrangular area,may be readily obtained by dividing the data of the first quadrangulararea by the area of the first quadrangular area.

Although FIG. 1 and the above-described embodiment express the processor120 and the DSP 190 as separate components, the processor 120 mayinclude the DSP 190. Accordingly, the processor 120 may perform thefollowing operations.

According to various embodiments of the present disclosure, theprocessor 120 calculates, for each pixel of the obtained image, anaccumulated-pixel value by adding up the pixel values of the pixels in aquadrangular area that takes a corresponding pixel and a reference pixelof the frame as corners; and generates an image qualityprocessing-dedicated frame of accumulated pixel values including thecalculated accumulated-pixel value corresponding to each pixel of theframe.

According to various embodiments of the present disclosure, theprocessor 120 may be configured to reduce the size of the generatedimage quality processing-dedicated frame using sub-sampling or binning.

According to various embodiments of the present disclosure, theprocessor 120 may reduce the size of the frame using sub-sampling orbinning.

According to various embodiments of the present disclosure, theprocessor 120 may perform data processing on the frame, so as togenerate image quality data associated with the frame.

According to various embodiments of the present disclosure, the imagequality data may include auto focus data, auto white balance data, orauto exposure data.

According to various embodiments of the present disclosure, theprocessor 120 may generate an image quality processing-dedicated framewith respect to the obtained image using the obtained image; and may seta first region of interest in the obtained image using the generatedimage quality processing-dedicated frame, or may set a second region ofinterest having the size or location that is different from at least oneof the size and location of the first region of interest.

The second region of interest may be set when the result of areliability test of the first region of interest shows that the firstregion of interest fails to satisfy a designated standard.

According to various embodiments of the present disclosure, theprocessor 120 may obtain the data of the first region of interest, whichincludes at least one pixel, from the generated image qualityprocessing-dedicated frame. The operation of obtaining the data of thefirst region of interest may include: adding a second pixel that is incontact in the diagonal direction with a pixel that takes, as aboundary, a second corner disposed in the diagonal direction to a firstcorner in the first region of interest, out of the pixels outside thefirst region of interest, to a pixel value of a first pixel in the firstregion of interest that takes the first corner, which is the farthestfrom the reference pixel in the first region of interest, as a boundary;subtracting, from the sum, a third pixel that is in contact in thediagonal direction with a pixel that takes, as a boundary, a thirdcorner in the first region of interest, out of the pixels outside thefirst region of interest; and subtracting, from the sum, a fourth pixelthat is in contact in the diagonal direction with a pixel that takes, asa boundary, a fourth corner in the first region of interest, out of thepixels outside the first region of interest.

According to various embodiments of the present disclosure, theprocessor 120 may obtain the data of the second region of interestincluding at least one pixel from the generated image qualityprocessing-dedicated frame. The operation of obtaining the data of thesecond region of interest may include: adding a sixth pixel that is incontact in the diagonal direction with a pixel that takes, as aboundary, a sixth corner disposed in the diagonal direction to a fifthcorner in the second region of interest, out of the pixels outside thesecond region of interest, to a pixel value of the fifth pixel in thesecond region of interest that takes the fifth corner, which is thefarthest from the reference pixel in the second region of interest, as aboundary; subtracting, from the sum, a seventh pixel that is in contactin the diagonal direction with a pixel that takes, as a boundary, aseventh corner in the second region of interest, out of the pixelsoutside the second region of interest; and subtracting, from the sum, aneighth pixel that is in contact in the diagonal direction with a pixelthat takes, as a boundary, an eighth corner in the second region ofinterest, out of the pixels outside the second region of interest.

FIG. 2 is a block diagram illustrating an electronic device 201according to various embodiments of the present disclosure. Theelectronic device 201 may include, for example, an entirety or part ofthe electronic device 101 illustrated in FIG. 1. The electronic device201 may include at least one application processor (AP) 210, acommunication module 220, a subscriber identification module (SIM) 224,a memory 230, a sensor module 240, an input device 250, a display 260,an interface 270, an audio module 280, a camera module 291, a powermanagement module 295, a battery 296, an indicator 297, and a motor 298.

The processor 210 may control a plurality of hardware or softwarecomponents connected to the processor 210 by driving an operating systemor an application program, and may perform various types of dataprocessing and calculations. The processor 210 may be embodied as, forexample, a System on Chip (SoC). According to an embodiment of thepresent disclosure, the processor 210 may further include a graphicprocessing unit (GPU) and/or an image signal processor. The processor210 may also include at least some (e.g., a cellular module 221) of thecomponents illustrated in FIG. 2. The processor 210 may load commands ordata received from at least one of the other components (e.g., anon-volatile memory) in a volatile memory, may process the loadedcommands or data, and may store various types of data in a non-volatilememory.

The communication module 220 may have a configuration that is equal orsimilar to that of the communication interface 170 of FIG. 1. Thecommunication module 220 may include, for example, a cellular module221, a Wi-Fi module 223, a Bluetooth module 225, a GNSS module 227(e.g., a GPS module, a Glonass module, a Beidou module, or a Galileomodule), an NFC module 228, and a radio frequency (RF) module 229.

For example, the cellular module 221 may provide a voice call, an imagecall, a text message service, an Internet service, and the like througha communication network. According to an embodiment of the presentdisclosure, the cellular module 221 may identify or authenticate anelectronic device 201 in the communication network by using thesubscriber identification module (e.g., a SIM card) 224. According to anembodiment of the present disclosure, the cellular module 221 mayperform at least some of the functions that the AP 210 may provide.According to an embodiment of the present disclosure, the cellularmodule 221 may include a communication processor (CP).

For example, each of the Wi-Fi module 223, the BT module 225, the GNSSmodule 227, and the NFC module 228 may include a processor forprocessing data transmitted/received through a corresponding module.According to embodiments of the present disclosure, at least some (twoor more) of the cellular module 221, the Wi-Fi module 223, the Bluetoothmodule 225, the GNSS module 227, and the NFC module 228 may be includedin a single Integrated Chip (IC) or IC package.

The RF module 229 may transmit/receive, for example, a communicationsignal (e.g., an RF signal). The RF module 229 may include, for example,a transceiver, a power amplifier module (PAM), a frequency filter, a lownoise amplifier (LNA), and an antenna. According to another embodimentof the present disclosure, at least one of the cellular module 221, theWi-Fi module 223, the Bluetooth module 225, the GNSS module 227, and theNFC module 228 may transmit/receive an RF signal through a separate RFmodule.

The subscriber identification module 224 may include, for example, acard including a subscriber identity module and/or an embedded SIM, andmay contain unique identification information (e.g., an integratedcircuit card identifier (ICCID)) or subscriber information (e.g., aninternational mobile subscriber identity (IMSI)).

The memory 230 (e.g., the memory 130) may include, for example, anembedded memory 232 or an external memory 234. The embedded memory 232may include at least one of, for example, a volatile memory (e.g., adynamic random access memory (DRAM), a static RAM (SRAM), a synchronousdynamic RAM (SDRAM), and the like) and a non-volatile memory (e.g., aone time programmable read only memory (OTPROM), a programmable ROM(PROM), an erasable and programmable ROM (EPROM), an electricallyerasable and programmable ROM (EEPROM), a flash memory (e.g., a NANDflash memory or a NOR flash memory), a hard driver, or a solid statedrive (SSD).

The external memory 234 may further include a flash drive, for example,a compact flash (CF), a secure digital (SD), a micro secure digital(Micro-SD), a mini secure digital (Mini-SD), an eXtreme digital (xD), amulti-media card (MMC), a memory stick, or the like. The external memory234 may be functionally and/or physically connected to the electronicdevice 201 through various interfaces.

The sensor module 240 may measure a physical quantity or detect anoperation state of the electronic device 201, and may convert themeasured or detected information into an electrical signal. The sensormodule 240 may include, for example, at least one of a gesture sensor240A, a gyro sensor 240B, an atmospheric pressure sensor 240C, amagnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, aproximity sensor 240G, a color sensor 240H (e.g., a red, green, blue(RGB) sensor), a biometric sensor 240I, a temperature/humidity sensor240J, a light sensor 240K, and a ultraviolet (UV) sensor 240M.Additionally or alternatively, the sensor module 240 may include, forexample, an E-nose sensor, an electromyography (EMG) sensor, anelectroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, aninfrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. Thesensor module 240 may further include a control circuit for controllingone or more sensors included therein. In some embodiments of the presentdisclosure, the electronic device 201 may further include a processor,which is configured to control the sensor module 240, as a part of theprocessor 210 or separately from the processor 210 in order to controlthe sensor module 240 while the processor 210 is in a sleep state.

The input device 250 may include, for example, a touch panel 252, a(digital) pen sensor 254, a key 256, or an ultrasonic input device 258.The touch panel 252 may use at least one of, for example, a capacitivetype, a resistive type, an infrared type, and an ultrasonic type.Further, the touch panel 252 may further include a control circuit. Thetouch panel 252 may further include a tactile layer to provide a tactilereaction to a user.

The (digital) pen sensor 254 may include, for example, a recognitionsheet, which is a part of the touch panel or separated from the touchpanel. The key 256 may include, for example, a physical button, anoptical key or a keypad. The ultrasonic input device 258 may detectultrasonic waves generated by an input tool through a microphone (e.g.,a microphone 288), and determine data corresponding to the detectedultrasonic waves.

The display 260 (e.g., the display 160) may include a panel 262, ahologram device 264, or a projector 266. The panel 262 may include aconfiguration that is identical or similar to that of the display 160illustrated in FIG. 1. The panel 262 may be embodied to be, for example,flexible, transparent, or wearable. The panel 262 may be formed, as asingle module, to be integrated with the touch panel 252. The hologramdevice 264 may show a three dimensional image in the air by using theinterference of light. The projector 266 may display an image byprojecting light onto a screen. The screen may be located, for example,inside or outside the electronic device 201. According to an embodimentof the present disclosure, the display 260 may further include a controlcircuit for controlling the panel 262, the hologram device 264, or theprojector 266.

The interface 270 may include, for example, a high-definition multimediainterface (HDMI) 272, a universal serial bus (USB) 274, an opticalinterface 276, or a D-subminiature (D-sub) 278. The interface 270 may beincluded in, for example, the communication interface 170 shown inFIG. 1. Additionally or alternatively, the interface 270 may include,for example, a mobile high-definition link (MHL) interface, a securedigital (SD) card/multi-media card (MMC) interface, or an infrared dataassociation (IrDA) standard interface.

The audio module 280 may bilaterally convert, for example, a sound andan electrical signal. At least some components of the audio module 280may be included in, for example, the input/output interface 150illustrated in FIG. 1. The audio module 280 may process soundinformation that is input or output through, for example, a speaker 282,a receiver 284, earphones 286, the microphone 288, or the like.

The camera module 291 is, for example, a device that may shoot a stillimage and a video. According to an embodiment of the present disclosure,the camera module 291 may include one or more image sensors (e.g., afront sensor or a back sensor), a lens, an Image Signal Processor (ISP)or a flash (e.g., LED or xenon lamp). The camera module 291 may include,for example, the camera module 180 of FIG. 1.

The power management module 295 may manage, for example, power of theelectronic device 201. According to an embodiment of the presentdisclosure, the power management module 295 may include a powermanagement integrated circuit (PMIC), a charger integrated circuit (IC),or a battery or fuel gauge. The PMIC may use a wired and/or wirelesscharging method. The wireless charging method may include, for example,a magnetic resonance scheme, a magnetic induction scheme, or anelectromagnetic scheme, and may further include an additional circuitfor wireless charging, such as a coil loop circuit, a resonance circuit,a rectifier circuit, and the like. The battery gauge may measure, forexample, a residual quantity of the battery 296, and a voltage, acurrent, or a temperature during the charging. The battery 296 mayinclude, for example, a rechargeable battery and/or a solar battery.

The indicator 297 may display a particular state, for example, a bootingstate, a message state, a charging state, or the like of the electronicdevice 201 or a part (e.g., the processor 210) of the electronic device201. The motor 298 may convert an electrical signal into mechanicalvibration, and may generate vibration, a haptic effect, or the like.Although not illustrated, the electronic device 201 may include aprocessing device (e.g., a GPU) for supporting a mobile TV. Theprocessing device for supporting a mobile TV may, for example, processmedia data according to a certain standard such as digital multimediabroadcasting (DMB), digital video broadcasting (DVB), or mediaFLO™.

Each of the above-described component elements of hardware according tothe present disclosure may be configured with one or more components,and the names of the corresponding component elements may vary based onthe type of electronic device. The electronic device according tovarious embodiments of the present disclosure may include at least oneof the aforementioned elements. Some elements may be omitted or otheradditional elements may be further included in the electronic device.Also, some of the hardware components according to various embodimentsmay be combined into one entity, which may perform functions identicalto those of the relevant components before the combination.

FIG. 3 is a block diagram of a program module according to variousembodiments of the present disclosure. According to an embodiment of thepresent disclosure, the program module 310 (e.g., the program 140) mayinclude an operating system (OS) that controls resources relating to anelectronic device (e.g., the electronic device 101) and/or variousapplications (e.g., the application programs 147) running on theoperating system. The operating system may be, for example, Android,iOS, Windows, Symbian, Tizen, Bada, or the like.

The program module 310 may include a kernel 320, middleware 330, anapplication programming interface (API) 360, and/or applications 370. Atleast some of the program module 310 may be preloaded in an electronicdevice or downloaded from an external electronic device (e.g., theelectronic device 102 or 104, or the server 106).

The kernel 320 (e.g., the kernel 141) may include, for example, a systemresource manager 321 and/or a device driver 323. The system resourcemanager 321 may control, allocate, or collect the system resources.According to an embodiment of the present disclosure, the systemresource manager 321 may include a process management unit, a memorymanagement unit, or a file system management unit. The device driver 323may include, for example, a display driver, a camera driver, a Bluetoothdriver, a shared-memory driver, a USB driver, a keypad driver, a Wi-Fidriver, an audio driver, or an Inter-Process Communication (IPC) driver.

The middleware 330, for example, may provide a function utilized by theapplications 370 in common or may provide various functions to theapplications 370 through the API 360 so that the applications 370 mayefficiently use limited system resources within the electronic device.According to an embodiment of the present disclosure, the middleware 330(e.g., the middleware 143) may include, for example, at least one of aruntime library 335, an application manager 341, a window manager 342, amultimedia manager 343, a resource manager 344, a power manager 345, adatabase manager 346, a package manager 347, a connectivity manager 348,a notification manager 349, a location manager 350, a graphic manager351, and a security manager 352.

The runtime library 335, for example, may include a library module,which a compiler uses in order to add a new function through aprogramming language while the applications 370 are being executed. Theruntime library 335 may perform input/output management, memorymanagement, or a function for an arithmetic function.

The application manager 341 may manage, for example, a life cycle of atleast one of the applications 370. The block manager 342 may managegraphical user interface (GUI) resources used in a screen. Themultimedia manager 343 may recognize formats utilized to reproducevarious media files, and may perform an encoding or decoding of themedia file by using a codec suitable for a corresponding format. Theresource manager 344 may manage resources such as a source code, amemory, and a storage space of at least one of the applications 370.

The power manager 345 may, for example, operate together with a basicinput/output system (BIOS) to manage a battery or power, and may providepower information utilized for the operations of the electronic device.The database manager 346 may generate, search for, or change a databaseto be used by at least one of the applications 370. The package manager347 may manage the installation or updating of an applicationdistributed in the form of a package file.

The connectivity manager 348 may manage wireless connection, forexample, Wi-Fi or Bluetooth. The notification manager 349 may display ornotify of an event such as a received message, an appointment, aproximity notification, and the like in such a way that does not disturba user. The location manager 350 may manage location information of theelectronic device. The graphic manager 351 may manage graphic effects tobe provided to a user or user interfaces related to the graphic effects.The security manager 352 may provide all security functions utilized forsystem security, user authentication, or the like. According to anembodiment of the present disclosure, when the electronic device (e.g.,the electronic device 101) has a telephone call function, the middleware330 may further include a telephony manager that manages a voice orvideo call function of the electronic device.

The middleware 330 may include a middleware module for forming acombination of various functions of the aforementioned components. Themiddleware 330 may provide a module specialized for each type ofoperating system in order to provide a differentiated function. Further,the middleware 330 may dynamically remove some of the existingcomponents or add new components.

The API 360 (e.g., the API 145) is, for example, a set of APIprogramming functions, and a different configuration thereof may beprovided according to an operating system. For example, one API set maybe provided for each platform in the case of Android or iOS, and two ormore API sets may be provided for each platform in the case of Tizen.

The applications 370 (e.g., the application programs 147) may include,for example, one or more applications that can perform functions, suchas home 371, dialer 372, SMS/MMS 373, Instant Message (IM) 374, browser375, camera 376, alarm 377, contacts 378, voice dialer 379, e-mail 380,calendar 381, media player 382, album 383, clock 384, health care (e.g.,measuring the amount of exercise or blood sugar), or environmentinformation (e.g., providing atmospheric pressure, humidity, temperatureinformation or the like).

According to an embodiment of the present disclosure, the applications370 may include an application (hereinafter, referred to as an“information exchanging application” for convenience of description)that supports exchanging information between the electronic device(e.g., the electronic device 101) and an external electronic device(e.g., the electronic device 102 or 104). The information exchangingapplication may include, for example, a notification relay applicationfor transferring predetermined information to an external electronicdevice, or a device management application for managing an externalelectronic device.

For example, the notification relay application may include a functionof delivering, to the external electronic device (e.g., the electronicdevice 102 or 104), notification information generated by otherapplications (e.g., an SMS/MMS application, an email application, ahealth care application, an environmental information application, andthe like) of the electronic device. Further, the notification relayapplication may receive notification information from, for example, anexternal electronic device and may provide the received notificationinformation to a user.

The device management application may manage (e.g., install, delete, orupdate), for example, at least one function of an external electronicdevice (e.g., the electronic device 102 or 104) that communicates withan electronic device (e.g., turning on/off the external electronicdevice itself (or some components thereof) or adjusting brightness (orresolution) of a display), applications executed in the externalelectronic device, or services provided in the external electronicdevice (e.g., a telephone call service or a message service).

According to an embodiment of the present disclosure, the applications370 may include applications (e.g., a health care application of amobile medical appliance, or the like) designated according to theattributes of an external electronic device (e.g., the electronic device102 or 104). According to an embodiment of the present disclosure, theapplications 370 may include applications received from an externalelectronic device (e.g., the server 106 or the electronic device 102 or104). According to an embodiment of the present disclosure, theapplications 370 may include a preloaded application or a third partyapplication that may be downloaded from a server. The names of thecomponents of the program module 310, according to the embodimentillustrated in the drawing, may vary according to the type of operatingsystem.

According to various embodiments of the present disclosure, at least apart of the programming module 310 may be embodied by software,firmware, hardware, or a combination of two or more thereof. At leastsome of the program module 310 may be implemented (e.g., executed) by,for example, the processor (e.g., the processor 210). At least some ofthe program module 310 may include, for example, a module, a program, aroutine, a set of instructions, a process, or the like, for performingone or more functions.

FIG. 4 is a flowchart illustrating an image processing operation of anelectronic device (e.g., the electronic device 101) according to variousembodiments of the present disclosure. FIGS. 5A and 5B are diagramsillustrating an image processing operation of an electronic device(e.g., the electronic device 101) according to various embodiments ofthe present disclosure.

In operation 410, the electronic device obtains an image through acamera module (e.g., capturing an image via the camera module 180).

In operation 430, the electronic device generates an image qualityprocessing-dedicated frame with respect to the obtained image. Forexample, the electronic device may calculate, for each pixel of theobtained image, an accumulated-pixel value by adding up the pixel valuesof pixels in a quadrangular area that utilizes a corresponding pixel anda reference pixel of the frame as corners. Also, the electronic devicemay generate an image quality processing-dedicated frame ofaccumulated-pixel values including the calculated accumulated-pixelvalue corresponding to each pixel of the frame.

Referring to FIGS. 5A and 5B, a method of generating the image qualityprocessing-dedicated frame is described as follows. From the obtainedimage, provided in a size of M×N including pixel values (I(x0, y0),I(x0, y1), . . . I(x1, y2)) of some pixels (as illustrated in FIG. 5A),an image quality processing-dedicated image (corresponding to theobtained image) may be generated with respect to the obtained image (asillustrated in FIG. 5B), by applying a method of “accumulatively” addingup pixel values according to (or based on) a predetermined condition. Byutilizing a pixel at (x0, y0) coordinate disposed in the obtained imageof FIG. 5A as a reference pixel 501, and gradually enlarging aquadrangular area from a single quadrangular area that includes thereference pixel 501, an accumulated-pixel value of a pixel at acoordinate in the image quality processing-dedicated frame may becalculated, which corresponds to a pixel value of a pixel at eachcoordinate in the obtained image that may be thus calculated.

For example, when the quadrangular area is formed of solely of thereference pixel 501 at the (x0, y0) coordinate, an accumulated-pixelvalue 502 for the pixel at a coordinate corresponding to the (x0, y0)coordinate in the image quality processing-dedicated frame is I(x0, y0).Also, when the quadrangular area is formed by utilizing the referencepixel 501 at the (x0, y0) coordinate and a pixel at a (x0, y1)coordinate as corners, an accumulated-pixel value of a pixel at acoordinate that corresponds to the (x0, y1) coordinate, in the imagequality processing-dedicated frame, is I(x0, y0)+I(x0, y1). Also, whenthe quadrangular area is formed by utilizing the reference pixel 501 atthe (x0, y0) coordinate and a pixel at a (x0, y2) coordinate as corners,an accumulated-pixel value of a pixel at a coordinate that correspondsto the (x0, y2) coordinate, in the image quality processing-dedicatedframe, is I(x0, y0)+I(x0, y1)+I(x0, y2). Also, when the quadrangulararea is formed by utilizing the reference pixel 501 at the (x0, y0)coordinate and a pixel at a (x1, y0) coordinate as corners, anaccumulated-pixel value of a pixel at a coordinate that corresponds tothe (x1, y0) coordinate, in the image quality processing-dedicatedframe, is I(x0, y0)+I(x1, y0). Also, when the quadrangular area isformed by utilizing the reference pixel 501 at the (x0, y0) coordinateand a pixel at a (x1, y1) coordinate as corners, an accumulated-pixelvalue of a pixel at a coordinate that corresponds to the (x1, y1)coordinate, in the image quality processing-dedicated frame, is I(x0,y0)+I(x0, y1)+I(x1, y0)+I(x1, y1). Also, when the quadrangular area isformed by utilizing the reference pixel 501 at the (x0, y0) coordinateand a pixel at a (x1, y2) coordinate as corners, an accumulated-pixelvalue of a pixel at a coordinate that corresponds to the (x1, y2)coordinate, in the image quality processing-dedicated frame, is I(x0,y0)+I(x0, y1)+I(x0, y2)+I(x1, y0)+I(x1, y1)+I(x1, y2).

By applying the above-described method, an accumulated-pixel value ofeach of the pixels of the remaining coordinates of the obtained imagemay be calculated. As illustrated in FIG. 5B, an image qualityprocessing-dedicated frame provided in the form of an image having asize of M×N may be generated.

According to various embodiments of the present disclosure, an operationof calculating the accumulated-pixel value may include an operation ofreducing the size of the obtained image by using sub-sampling (alsoknown as down-sampling or decimation) or binning. For example, theoperation of calculating the accumulated-pixel value may be applied towhich the size is reduced.

According to various embodiments of the present disclosure, theoperation of calculating the accumulated pixel value may include anoperation of performing data processing on the obtained image so as togenerate image quality data for the obtained image. For example, theoperation of calculating the accumulated-pixel value may be applied tothe data-processed frame. The operation of performing data processing onthe obtained image has been described, and thus, the detaileddescriptions thereof will be omitted.

According to various embodiments of the present disclosure, theelectronic device (e.g., the electronic device 101) may obtain data of apredetermined area (hereinafter, referred to as a first quadrangulararea), which includes at least one pixel, from the generated imagequality processing-dedicated frame.

The operation of obtaining the data of the first quadrangular area maybe performed through, for example, adding a second pixel that is incontact in the diagonal direction with a pixel that takes, as aboundary, a second corner disposed in the diagonal direction to a firstcorner in the first quadrangular area, out of the pixels outside thefirst quadrangular area, to a pixel value of a first pixel in the firstquadrangular area that takes the first corner, which is the farthestfrom the reference pixel in the first quadrangular area, as a boundary;subtracting, from the sum, a third pixel that is in contact in thediagonal direction with a pixel that takes, as a boundary, a thirdcorner in the first quadrangular area, out of the pixels outside thefirst quadrangular area; and subtracting, from the sum, a fourth pixelthat is in contact in the diagonal direction with a pixel that takes, asa boundary, a fourth corner in the first quadrangular area, out of thepixels outside the first quadrangular area.

Referring to FIGS. 5A and 5B, a method of obtaining the data of thefirst quadrangular area will be described as follows.

For example, the data of the first quadrangular area 504 of the imagequality processing-dedicated frame, which corresponds to a quadrangulararea 503 including pixels at (x1, y1) and (x1, y2) coordinates, may beobtained as follows. When the pixel at the coordinate (x0, y0) is set asthe reference pixel 501, the pixel value of pixel 502 (e.g., (x0, y0) isadded to the pixel value of pixel 507 (e.g., (x1, y2)) to form a sum.Then, a pixel value of pixel 505 (e.g., (x0, y2)) and a pixel value ofpixel 506 (e.g., (x1, y2) are subtracted from the sum, resulting in thedata of the first quadrangular area 504 of the image qualityprocessing-dedicated frame may be obtained.

The data of the first quadrangular area 504 of the image qualityprocessing-dedicated frame may be determined as follows, with referenceto FIG. 5B:

-   {I(x0, y0)+I(x0, y1)+I(x0, y2)+I(x1, y0)+I(x1, y1)+I(x1, y2)} (e.g.,    pixel value at 507)-   +I(x0, y0) (e.g. pixel value at 502)-   −{I(x0, y0)+I(x0, y1)+I(x0, y2)} (e.g., pixel value at 505)-   −{I(x0, y0)+I(x1, y0)} (e.g. pixel value at 506)-   =I(x1, y1)+I(x1, y2).

According to the above described calculation, it can be recognized thatthe data of the first quadrangular area 504 of the image qualityprocessing-dedicated frame is a value that is equal to image qualityprocessing-dedicated data obtained by summing the pixel values of thecorresponding quadrangular area 503 of the obtained image.

Referring to FIG. 6, a method of obtaining the data of a predeterminedarea of an image quality processing-dedicated frame may be expressed asan equation. The data of a quadrangular area formed of A, B, C, and D inan image 601 may be obtained using Equation 1, as provided below:

Value in ABCD=ACC[A]−ACC[B]−ACC[C]+ACC[D]   [Equation 1]

“Value in ABCD” is the data (or data value) of a quadrangular areaformed of A, B, C, and D. ACC[A] indicates the data of a pixel at apoint A of an image quality processing-dedicated frame (e.g., the pixelvalue of a pixel in the quadrangular area that utilizes the point A asone of four corners). ACC[B] indicates the data of a pixel at a point Bof the image quality processing-dedicated frame (e.g., the pixel valueof a pixel in the quadrangular area that utilizes the point B as acorner). ACC[C] indicates the data of a pixel at a point C of the imagequality processing-dedicated frame (e.g., the pixel value of a pixel inthe quadrangular area that utilizes the point C as a corner). ACC[D]indicates the data of a pixel at a point D of the image qualityprocessing-dedicated frame (e.g., the pixel value of a pixel in thequadrangular area that utilizes the point D as a corner).

FIG. 7 is a flowchart illustrating an image processing operation of anelectronic device (e.g., the electronic device 101) according to variousembodiments of the present disclosure.

In operation 710, the electronic device obtains an image through acamera module (e.g., capturing the image via the camera module 180).

In operation 730, to generate image quality data of the obtained image,the electronic device performs data processing on the frame.

In operation 750, the electronic device generates an image qualityprocessing-dedicated frame 790 by accumulatively adding up pixel valuesbased on a predetermined condition in the data-processed frame.

Operations 730 to 750 may be respectively identical to the dataprocessing operation and the operation of generating an image qualityprocessing-dedicated frame based on an accumulative summation, whichhave been described with reference to FIGS. 4 to 6.

FIG. 8 is a flowchart illustrating an image processing operation of anelectronic device (e.g., the electronic device 101) according to variousembodiments of the present disclosure.

In operation 810, the electronic device obtains an image through acamera module (e.g., captured via the camera module 180).

In operation 830, to generate image quality data of the obtained image,the electronic device performs data processing on the frame.

In operation 850, the electronic device generates a basic image qualityprocessing-dedicated frame 860 (e.g., the image qualityprocessing-dedicated frame 790) by accumulatively adding up pixel valuesbased on a predetermined condition in the data-processed frame. Forexample, referring to FIG. 9, a basic image quality processing-dedicatedframe 960 (Acclmage (M,N); e.g., the basic image qualityprocessing-dedicated frame 860) may have a size of M×N. The size may beequal to that of the obtained image.

In the same manner as operations 730 to 750, operations 830 to 850 maybe respectively identical to the data processing operation and theoperation of generating an image quality processing-dedicated framebased on an accumulative summation, which have been described withreference to FIGS. 4 to 6.

In operation 870, the electronic device reduces the size of thegenerated basic image quality processing-dedicated frame 860 usingbinning, so as to generate the image quality processing-dedicated frame890 for which the size is reduced. For example, referring to FIG. 10, areduced image quality processing-dedicated frame 1090 (Acclmage(X,Y));e.g., the image quality processing-dedicated frame 890) may be generatedby reducing the basic image quality processing-dedicated frame (e.g.,the basic image quality processing-dedicated frame 860) provided in asize of M×N to a size of X×Y.

According to various embodiments of the present disclosure, an imageprocessing method of an electronic device may include: obtaining animage through a camera module of the electronic device; generating animage quality processing-dedicated frame with respect to the obtainedimage using the obtained image; and setting a first region of interestin the obtained image or setting a second region of interest having asize or location that is different from at least one of a size andlocation of the first region of interest, by using the generated imagequality processing-dedicated frame.

The data of the first region of interest is obtained by: adding a secondpixel that is in contact in the diagonal direction with a pixel thattakes, as a boundary, a second corner disposed in the diagonal directionto a first corner in the first region of interest, out of the pixelsoutside the first region of interest, to a pixel value of a first pixelin the first region of interest that takes the first corner, which isthe farthest from the reference pixel in the first region of interest,as a boundary; subtracting, from the sum, a third pixel that is incontact in the diagonal direction with a pixel that takes, as aboundary, a third corner in the first region of interest, out of thepixels outside the first region of interest; and subtracting, from thesum, a fourth pixel that is in contact in the diagonal direction with apixel that takes, as a boundary, a fourth corner in the first region ofinterest, out of the pixels outside the first region of interest.

The data of the second region of interest is obtained by: adding a sixthpixel that is in contact in the diagonal direction with a pixel thattakes, as a boundary, a sixth corner disposed in the diagonal directionto a fifth corner in the second region of interest, out of the pixelsoutside the second region of interest, to a pixel value of a fifth pixelin the second region of interest that takes, as a boundary, the fifthcorner which is the farthest from the reference pixel in the secondregion of interest; subtracting, from the sum, a seventh pixel that isin contact in the diagonal direction with a pixel that takes, as aboundary, a seventh corner in the second region of interest, out of thepixels outside the second region of interest; and subtracting, from thesum, an eighth pixel that is in contact in the diagonal direction with apixel that takes, as a boundary, an eighth corner in the second regionof interest, out of the pixels outside the second region of interest.

According to the above described embodiments of the present disclosure,an image quality processing-dedicated frame with respect to an imageshot by a camera is generated, and by using the generated image qualityprocessing-dedicated frame, image processing for improving the imagequality of an image obtained by the camera (or for generating imagequality data of an image obtained through the camera) may be efficientlyperformed.

For example, referring to FIG. 11, when an image qualityprocessing-dedicated frame exists, which is generated with respect to animage obtained through a camera, the 3A algorithm for image qualityprocessing may be accurately controlled. For example, when a first block111 is set in the image of FIG. 11 by predicting an area using data ofimage quality processing executed in advance, a block 113 and 117 may begenerated by enlarging or reducing the area of the first block 111 usingthe generated image quality processing-dedicated frame, or a block 115may be newly generated in another area, and thus, data in a block may bereadily obtained. Accordingly, the 3A algorithm may be implementedflexibly.

Also, for example, referring to FIG. 12, in the situation in which aface in an image obtained through a camera is recognized using a facialrecognition technology, when the coordinates of a recognized area (e.g.,where first block 121 is to be set) are provided, the recognized area121 may be set as a first block 121 and image processing may beperformed using the data of the first block 121. However, according toembodiments of the present disclosure, the reliability of the firstblock 121 is attempted to be determined, and the reliability of a secondblock 123 generated through enlarging the area of the first block 121 isattempted to be determined, and/or the reliability of a third block 125generated through reducing the area of the first block 121 is attemptedto be determined. These may be used to increase the accuracy of the 3Aalgorithm.

Also, for example, when a video is captured using a camera, asillustrated in FIG. 13, it is beneficial to quickly and accuratelydetect a block (or a plurality of blocks 131 and 133) to track asubject. Generally, to track a subject in live video via a camera, onemay predict a block of an image to be subsequently obtained, or predictthe movement of the subject to detect the location of a focus of theimage. In one example, the data of a block is detected in at least 2VDtime after the block is set, and thus, errors easily occur in setting anarea. However, according to the embodiments of the present disclosure,when a predicted block area is an incorrectly predicted area, it ispossible to quickly move an area and to apply an image qualityalgorithm, allowing a subject to be quickly and accurately tracked.

FIG. 14 is a diagram illustrating an operating time of an imageprocessing operation of an electronic device according to variousembodiments of the present disclosure. As seen therein, according to atime (T), a vertical drive (VD) signal is generated after lightexposure. After the VD signal, data is processed and obtained.

The term “module” as used herein may, for example, mean a unit includingone of hardware, software, and firmware or a combination of two or moreof them. The “module” may be interchangeably used with, for example, theterm “unit”, “logic”, “logical block”, “component”, or “circuit”. The“module” may be a minimum unit of an integrated component element or apart thereof. The “module” may be a minimum unit for performing one ormore functions or a part thereof. The “module” may be mechanically orelectronically implemented. For example, the “module” according to thepresent disclosure may include at least one of an Application-SpecificIntegrated Circuit (ASIC) chip, a Field-Programmable Gate Arrays (FPGA),and a programmable-logic device for performing operations which has beenknown or are to be developed hereinafter.

According to various embodiments, at least some of the devices (forexample, modules or functions thereof) or the method (for example,operations) according to the present disclosure may be implemented by acommand stored in a computer-readable storage medium in a programmingmodule form. The instruction, when executed by a processor (e.g., theprocessor 120), may cause the one or more processors to execute thefunction corresponding to the instruction. The computer-readable storagemedium may be, for example, the memory 130.

The computer readable recoding medium may include a hard disk, a floppydisk, magnetic media (e.g., a magnetic tape), optical media (e.g., aCompact Disc Read Only Memory (CD-ROM) and a Digital Versatile Disc(DVD)), magneto-optical media (e.g., a floptical disk), a hardwaredevice (e.g., a Read Only Memory (ROM), a Random Access Memory (RAM), aflash memory), and the like. In addition, the program instructions mayinclude high class language codes, which can be executed in a computerby using an interpreter, as well as machine codes made by a compiler.The aforementioned hardware device may be configured to operate as oneor more software modules in order to perform the operation of thepresent disclosure, and vice versa.

The programming module according to the present disclosure may includeone or more of the aforementioned components or may further includeother additional components, or some of the aforementioned componentsmay be omitted. Operations executed by a module, a programming module,or other component elements according to various embodiments of thepresent disclosure may be executed sequentially, in parallel,repeatedly, or in a heuristic manner. Furthermore, some operations maybe executed in a different order or may be omitted, or other operationsmay be added. Various embodiments disclosed herein are provided merelyto easily describe technical details of the present disclosure and tohelp the understanding of the present disclosure, and are not intendedto limit the present disclosure. Therefore, it should be construed thatall modifications and changes or modified and changed forms based on thetechnical idea of the present disclosure fall within the presentdisclosure.

The above-described embodiments of the present disclosure can beimplemented in hardware, firmware or via the execution of software orcomputer code that can be stored in a recording medium such as a CD ROM,a Digital Versatile Disc (DVD), a magnetic tape, a RAM, a floppy disk, ahard disk, or a magneto-optical disk or computer code downloaded over anetwork originally stored on a remote recording medium or anon-transitory machine readable medium and to be stored on a localrecording medium, so that the methods described herein can be renderedvia such software that is stored on the recording medium using a generalpurpose computer, or a special processor or in programmable or dedicatedhardware, such as an ASIC or FPGA. As would be understood in the art,the computer, the processor, microprocessor controller or theprogrammable hardware include memory components, e.g., RAM, ROM, Flash,etc. that may store or receive software or computer code that whenaccessed and executed by the computer, processor or hardware implementthe processing methods described herein.

The control unit may include a microprocessor or any suitable type ofprocessing circuitry, such as one or more general-purpose processors(e.g., ARM-based processors), a Digital Signal Processor (DSP), aProgrammable Logic Device (PLD), an Application-Specific IntegratedCircuit (ASIC), a Field-Programmable Gate Array (FPGA), a GraphicalProcessing Unit (GPU), a video card controller, etc.

In addition, it would be recognized that when a general purpose computeraccesses code for implementing the processing shown herein, theexecution of the code transforms the general purpose computer into aspecial purpose computer for executing the processing shown herein. Anyof the functions and steps provided in the Figures may be implemented inhardware, software or a combination of both and may be performed inwhole or in part within the programmed instructions of a computer. Noclaim element herein is to be construed under the provisions of 35U.S.C. 112, sixth paragraph, unless the element is expressly recitedusing the phrase “means for”. In addition, an artisan understands andappreciates that a “processor” or “microprocessor” may be hardware inthe claimed disclosure. Under the broadest reasonable interpretation,the appended claims are statutory subject matter in compliance with 35U.S.C. §101.

What is claimed is:
 1. An electronic device, comprising: a camera modulethat obtains an image; and a processor, wherein the processor isconfigured to perform: setting a quadrangular area in the obtained imageincluding a reference pixel and a corresponding pixel disposedrespectively at corners of the quadrangular area, calculating anaccumulated-pixel value for each pixel of the obtained imagecorresponding to the quadrangular area, such that a particular pixelvalue for a particular pixel is a sum of the pixel values beginning fromthe reference pixel, continuing through an arrangement of pixels in thequadrangular area and terminating at the particular pixel; andgenerating an image quality processing-dedicated frame ofaccumulated-pixel values based on calculated accumulated-pixel values ofeach pixel of the frame.
 2. The electronic device of claim 1, whereinthe processor is configured to reduce a size of the generated imagequality processing-dedicated frame using at least one of sub-samplingand binning.
 3. The electronic device of claim 1, wherein the processoris configured to reduce a size of the frame using at least one ofsub-sampling and binning.
 4. The electronic device of claim 1, whereinthe processor is configured to perform data processing on the frame togenerate image quality data associated with the frame.
 5. The electronicdevice of claim 4, wherein the image quality data includes at least oneof auto focus (AF) data, auto white balance (AWB) data, and autoexposure (AE) data.
 6. An image processing method for an electronicdevice, comprising: obtaining an image through a camera module of theelectronic device; setting a quadrangular area in the obtained imageincluding a reference pixel and a corresponding pixel disposedrespectively at corners of the quadrangular area; calculating anaccumulated-pixel value for each pixel of the obtained imagecorresponding to the quadrangular area, such that a particular pixelvalue for a particular pixel is a sum of the pixel values beginning fromthe reference pixel, continuing through an arrangement of pixels in thequadrangular area and terminating at the particular pixel; andgenerating an image quality processing-dedicated frame ofaccumulated-pixel values based on calculated accumulated-pixel values ofeach pixel of the frame.
 7. The method of claim 6, further comprising:reducing a size of the generated image quality processing-dedicatedframe using at least one of sub-sampling and binning.
 8. The method ofclaim 6, wherein the calculating of the accumulated-pixel valuecomprises: reducing a size of the frame using at least one ofsub-sampling and binning.
 9. The method of claim 6, wherein thecalculating of the accumulated-pixel value comprises: performing dataprocessing on the frame to generate image quality data associated withthe frame.
 10. The method of claim 9, wherein the image quality dataincludes at least one of auto focus (AF) data, auto white balance (AWB)data, and auto exposure (AE) data.
 11. An electronic device, comprising:a camera module that obtains an image; and a processor, configured toperform: generating an image quality processing-dedicated frame withrespect to the obtained image based on the obtained image; and settingat least one of: a first region of interest disposed within the obtainedimage using the generated image quality processing-dedicated frame, anda second region of interest having a size or disposed at a locationdiffering from at least one of a size and location of the first regionof interest, respectively.
 12. The electronic device of claim 11,wherein the processor is configured to perform: setting a quadrangulararea in the obtained image including a reference pixel and acorresponding pixel disposed respectively at corners of the quadrangulararea, calculating, for each pixel of the frame, an accumulated-pixelvalue such that a particular pixel value for a particular pixel is a sumof the pixel values beginning from the reference pixel, continuingthrough an arrangement of pixels in the quadrangular area andterminating at the particular pixel; and generating the image qualityprocessing-dedicated frame including accumulated-pixel values based oncalculated accumulated-pixel values for each pixel of the obtainedimage.
 13. The electronic device of claim 12, wherein the processor isconfigured to reduce a size of the generated image qualityprocessing-dedicated frame using at least one of sub-sampling andbinning.
 14. The electronic device of claim 12, wherein the first regionof interest includes the quadrangular area, and the processor isconfigured to obtain data of the first region of interest from thegenerated image quality processing-dedicated frame by: adding a firstpixel value of a first pixel disposed at a first corner of thequadrangular area to a second pixel value of a second pixel disposed ata second corner of the quadrangular area, the first corner diagonallyopposite the second corner, and the first pixel disposed farther fromthe reference pixel among all pixels in the first region of interest,and subtracting, from the sum of the first pixel value and the secondpixel value, both a third pixel value of a third pixel disposed at athird corner of the quadrangular area and a fourth pixel value of afourth pixel disposed at a fourth corner of the quadrangular area. 15.The electronic device of claim 12, wherein second region of interestincludes the quadrangular area, and the processor is configured toobtain data of the second region of interest from the generated imagequality processing-dedicated frame by: adding a first pixel value of afirst pixel disposed at a first corner of the quadrangular area to asecond pixel value of a second pixel disposed at a second corner of thequadrangular area, the first corner diagonally opposite the secondcorner, and the first pixel disposed farther from the reference pixelamong all pixels in the first region of interest, and subtracting, fromthe sum of the first pixel value and the second pixel value, both athird pixel value of a third pixel disposed at a third corner of thequadrangular area and a fourth pixel value of a fourth pixel disposed ata fourth corner of the quadrangular area.
 16. An image processing methodof an electronic device, comprising: obtaining an image through a cameramodule of the electronic device; generating an image qualityprocessing-dedicated frame with respect to the obtained image based onthe obtained image; and setting at least one of: a first region ofinterest disposed within the obtained image using the generated imagequality processing-dedicated frame, and a second region of interesthaving a size or disposed at a location differing from at least one of asize and location of the first region of interest, respectively.
 17. Themethod of claim 16, wherein the generating of the image qualityprocessing-dedicated frame comprises: setting a quadrangular area in theobtained image including a reference pixel and a corresponding pixeldisposed respectively at corners of the quadrangular area, calculating,for each pixel of the frame, an accumulated-pixel value such that aparticular pixel value for a particular pixel is a sum of the pixelvalues beginning from the reference pixel, continuing through anarrangement of pixels in the quadrangular area and terminating at theparticular pixel; and generating the image quality processing-dedicatedframe including accumulated-pixel values based on calculatedaccumulated-pixel values for each pixel of the frame.
 18. The method ofclaim 17, further comprising: reducing the size of the generated imagequality processing-dedicated frame using at least one of sub-samplingand binning.
 19. The method of claim 17, wherein the first region ofinterest includes the quadrangular area, and data of the first region ofinterest is obtained from the generated image qualityprocessing-dedicated frame by: adding a first pixel value of a firstpixel disposed at a first corner of the quadrangular area to a secondpixel value of a second pixel disposed at a second corner of thequadrangular area, the first corner diagonally opposite the secondcorner, and the first pixel disposed farther from the reference pixelamong all pixels in the first region of interest, and subtracting, fromthe sum of the first pixel value and the second pixel value, both athird pixel value of a third pixel disposed at a third corner of thequadrangular area and a fourth pixel value of a fourth pixel disposed ata fourth corner of the quadrangular area.
 20. The method of claim 17,wherein second region of interest includes the quadrangular area, andthe data of the second region of interest is obtained from the generatedimage quality processing-dedicated frame by: adding a first pixel valueof a first pixel disposed at a first corner of the quadrangular area toa second pixel value of a second pixel disposed at a second corner ofthe quadrangular area, the first corner diagonally opposite the secondcorner, and the first pixel disposed farther from the reference pixelamong all pixels in the first region of interest, and subtracting, fromthe sum of the first pixel value and the second pixel value, both athird pixel value of a third pixel disposed at a third corner of thequadrangular area and a fourth pixel value of a fourth pixel disposed ata fourth corner of the quadrangular area.